inscatolati - Get out of the box! :)

If you run the "mount" command alone, you should get the list of the mounted partitions, something like:

/dev/sda1 on / type ext3 (rw,errors=remount-ro) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw)

Sometimes, especially when the root partition contains some errors, or has been mounted/remounted read only, the output of mount is screwd up: read-only partitions are shown as being read-write, unmounted partitions are still shown as mounted, and so on.

The problem is due to the fact that mount uses the file /etc/mtab for book keeping, writing there whenever a partition is mounted, unmounted, or remounted with different options.

If /etc/mtab is read only, for example because the root partition is read only, mount and many system tools may get confused about the status of the partitions, and may not be able to update their status. Symptoms are: partitions being unmounted without errors and still being shown by the mount command, read-only partitions shown as read-write, and all kind of inconsistencies...

To get a clean and correct view of the mounted partitions, you should take a look to the file /proc/mounts, with something like "cat /proc/mounts". That file shows the status of the partitions as seen by the kernel, and is correct most of the times.

Note, however, that on recent kernels (2.6.x and greater), every process may have a different "view" of which file systems are mounted and unmounted. If the situation gets that much confusing, one way to understand if a partition is read-write or read-only, is just to try to "touch" a simple file... have fun :)

If you want to change the partition table of a given device on a running system, without rebooting, just go on and modify the partition table, paying attention not to overwrite or shrink any used partition.

Once the partition table has been changed, you need to ask the kernel to reload the partition table and update the devices accordingly. To do so, you can run the command:

# sfdisk -R /dev/sda

where sda is the name of the device where the partition table was changed. If you get the error:

BLKRRPART: Device or resource busy

it means that /dev/sda, or the device you are using, has partitions mounted on it, and the kernel was unable to read the new table. If you want to use the new partition table without rebooting, you need to umount all the old partitions of /dev/sda and then re-run 'sfdisk -R'.

If 'sfdisk -R' returns no error, it means the new partition table was read and that the devices /dev/sda1, /dev/sda2... are now relative to the new partition table you just updated.

If you use LVM over RAID 5 on a 2.4 kernels, you might easily have errors like:

raid5: switching cache buffer size, 4096 --> 1024 raid5: switching cache buffer size, 1024 --> 4096 [...]

These messages are repeated almost everytime there is a disk access, flooding both your console, your logs, and degrading performance.

A quick and easy hack would be to create the filesystem with a block size of 4096. In the case of ext3, something like:

# mkfs.ext2 -b 4096 /dev/mapper/vg00-lv00

With this option, you will see your logs being flooded only when you are using low level tools, like 'vgs', 'lvs', 'tune2fs', and so on...

With other file systems, there are equivalent options to set the block size to 4096 bytes...

On Windows 2000, if you boot the PC with your poor, simple, 56k modem turned off... you loose the ability to use the modem!!

Solutions: reinstall the driver of the modem, or simply reboot your pc with the modem turned on. This time, it will properly work...

By default, the linux kernel will output lot of different kind of messages on your console. To disable them, you can run:

dmesg -n1

to tell the kernel to only send panic messages to your console. This is useful, for example, when you have enabled log martian in the kernel, added some -j LOG in iptables, or you have some hardware problem that's causing the kernel to spam you.

If you don't have a prompt, or it's too hard to write, and have emergency sysctl enabled in your kernel, you can try pressing ctrl+alt+stamp+1 (aka ctrl+alt+sysreq+1).

If you want this parameter to be automatically set at every reboot, you can add the -c1 parameter to klogd, in /etc/init.d. To do so in Debian, you need to modify /etc/default/klogd, to have something like:

KLOGD="-c1 [whatever parameter was already there]"

Note that even though this parameter is set to 1, the messages will still go to syslogd, which can decide that the message is important enough to be outputted in your console. Either stop syslogd temporarily, or change /etc/syslog.conf if it bothers you.

initrds created with mkinitramfs, mkinitrd, yaird or any other tool can sometime contain errors that make your system un-bootable, or that output error during the boot process. It is usually a pain to debug those errors, as... you often don't have a shell, needed softwares are missing from the initrd, ...

Generally, there are two approaches that can be easily used to debug an initrd:

• uncompress the initrd, and have a peek in the scripts to see what's being done and what it is doing when the init process stops (and why) - always works.

• at boot time, have the initrd output some debugging lines, or get a prompt to try to manually understand what's going wrong or why the commands are failing. Using this method requires support from the tool used to create the initrd, so it won't be discussed in this note.

So, to access the content of an initrd, you need to run a few commands depending on the format of the initrd itself. Nowdays, most initrds are either gzip-compressed cpio files, cramfs or other more or less esotheric file systems. You can start with something like:

% file -Ls /boot/initrd.img

If you are lucky, it will be a cramfs:

/boot/initrd.img-2.6.8-3-686-smp: Linux Compressed ROM File System data, [...]

just mount it with something like:

% mount -o loop /boot/initrd.img /mnt/whatever

If you are a bit less lucky, it will be gzip compressed:

/boot/initrd.img: gzip compressed data, from Unix,

Start by just uncompressing it:

% gzip -cd < /boot/initrd.img > /tmp/initrd.uncompressed

Repeat the "file" command above against /tmp/initrd.uncompressed. If you are lucky, again, it will be a filesystem. Just mount it with the same "mount -o loop..." as above. If you are a bit less lucky, you will see something like:

$ file -sL /tmp/initrd.uncompressed /tmp/initrd.uncompressed: ASCII cpio archive (SVR4 with no CRC)

which means that the initrd is a simple cpio archive. Uncompress it with something like:

$ cpio --extract --make-directories < /tmp/initrd.uncompressed

Now, you can start your debugging by either looking into /linuxrc, /init, or /sbin/init. You can obviously modify the filesystem, and reverse the steps to create a new initrd to test. Good luck!

mkinitramfs allows you to specify some parameters on the LILO or GRUB prompt to easily (sure) debug problems. Most useful parameters are probably:

• debug, to have all the shell scripts on the initrd run with the -x parameter, and the output logged in /tmp/initramfs.debug on the ramdisk. To have the output sent to stdout, specify something like debug=/dev/console.

• break, to have the initrd return a shell prompt. If no parameters are specified, the prompt will be returned when most convenient to the initrd (at current time, just before mounting the root filesystem - premount). Otherwise, you can specify something like break=whatever, to ask the initrd to stop at exactly the specified step. Steps currently defined by mkinitramfs are: top, modules, premount, mount, bottom and init.

• blacklist, if you suspect a module is causing problems to your hardware. Specify something like blacklist="module1 module2" to have module1 and module2 not loaded at boot time.

As I use grub on my system, to specify those parameter at boot time, I usually press 'e' on the line I usually boot from. Once there, I modify the line:

kernel /vmlinuz root=/dev/mapper/root ro

to have the options I need:

kernel /vmlinuz root=/dev/mapper/root ro break=top

and finally press 'b' to get the system booted with the specified parameters.